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Abstract-  A  computer-assisted  endoscopic  analysis  is  intended 
and  facilitates  the  diagnosis  process.  Segmentation  of  the  image 
is  an  important  step  and  a  novel  approach  is  proposed  to 
segment  clinical  endoscopic  images  based  on  homogeneity  and 
color  feature  hue.  In  the  first  stage,  the  regions  are  segmented 
using  a  peak-finding  algorithm  on  a  2-D  histogram  of 
homogeneity  and  intensity  values.  In  the  second  stage, 
histogram  analysis  of  the  color  feature  hue  is  performed  to 
subdivide  the  segmented  regions  obtained  from  the  first  stage. 
The  subdivisions  of  different  segmented  regions  having  similar 
CIE  (L*  a*  b*)  color  measure  are  merged.  The  proposed 
scheme  was  evaluated  on  a  database  of  clinical  endoscopic 
images. 

Keywords-  clinical  endoscopic  image,  color  image  segmentation, 
homogeneity,  color  space,  region  merging. 

I.  Introduction 

Minimally  invasive  endoscopic  procedures  are  increasingly 
employed  for  diagnostic  and  surgical  purposes  in  cases  such 
as  the  gastrointestinal  and  respiratory  ailments.  An 
endoscopist,  who  analyzes  the  acquired  images,  performs 
these  procedures.  A  computer-aided  scheme  will  help 
considerably  in  the  image  analysis  and  quantitative 
characterization  of  abnormalities,  thereby  improving  the 
overall  efficiency  of  the  diagnosis.  A  crucial  step  in  such  an 
automated  diagnostic  scheme  is  proper  segmentation  of  the 
endoscopic  images. 

Color  of  an  image  carries  much  more  information  than 
the  gray  levels.  In  many  pattern  recognition  and  computer 
vision  applications,  the  additional  information  provided  by 
color  can  help  in  image  analysis  and  provide  better  results 
than  approaches  using  merely  gray  scale  information. 

The  scales  employed  to  differentiate  pixels  in  a  color 
image  include  uniform  chromaticity  scale  (UCS),  OHTA, 
and  Hue,  Saturation,  and  Intensity  (HSI). 

The  UCS  approximates  human  perceptions  when  a 
Euclidean  distance  measure  in  the  (L*a*b*),  and 
(L*u*v*)  color  spaces  is  used  to  differentiate  between 
pixels  compare  to  (R,  G,  B)  and  (X,  Y,  Z)  color  spaces. 
The  UCS  matches  the  sensitivity  of  human  eye,  and  can 
mimic  human  perception. 

OHTA  color  space  was  derived  as  a  result  of  a  search  of 
completely  statistically  independent  components  on  a 
representative  sample  of  images.  The  OHTA  components 
are  good  approximations  of  the  results  of  the  Karhunen- 
Loeve  transformation,  which  is  very  good  with  respect  to 
decorrelation  of  RGB  components  [3]. 

HSI  uses  psychological  attributes  instead  of  human 
perceptions  to  differentiate  pixels.  The  perceptual  color 


space  is  used  to  capture  the  psychological  attributes,  and  it 
usually  described  by  Hue,  Saturation,  and  Intensity  (HSI). 
The  hue  is  the  attribute  of  color  perception  denoted  by  red, 
yellow,  green,  blue,  and  so  on.  Saturation  is  used  to  describe 
how  pure  a  color  is  or  how  much  white  is  added  to  a  pure 
color.  The  intensity  is  a  measure  of  total  reflectance  in  the 
visible  region  of  spectrum,  and  it  is  an  achromatic 
component  of  color.  Using  HSI  color  space  for  color  image 
segmentation  has  two  advantages:  (1)  specifying  and 
controlling  color  is  more  suitable  for  human  perception  than 
using  the  primary  colors  RGB;  (2)  it  can  control  intensity 
and  chromatic  components  more  easily  and  independently 
[1]. 

The  segmentation  of  the  image  is  performed  on  the  basis 
of  one  of  the  above  scales  using  histogram  analysis,  edge 
detection,  and  statistical  analysis. 

X.  Yang  [2]  developed  a  frizzy  logic  based  segmentation 
scheme  in  Hue,  Saturation,  and  Intensity  (HSI)  color  space. 
The  segmentation  was  realized  in  two  stages.  In  the  first 
stage,  a  3-D  histogram  of  HSI  color  space  was  obtained. 
Using  scale-space  filter,  a  number  of  regions  in  3-D  HSI 
space  were  segmented.  In  the  second  stage,  these  regions 
were  merged  using  fuzzy  logic  on  the  basis  of  their  edge 
strength  along  the  boundary,  color  similarity,  and  spatial 
connectivity  between  adjoining  regions. 

X.Y.  Zue  [3]  developed  two  stages  segmentation  scheme 
in  OHTA  color  space.  The  first  stage  is  similar  to  that 
proposed  by  X.  Yang  [2].  In  the  second  stage,  the  regions 
obtained  in  the  first  stage  were  merged  using  a  Markov 
Random  Field  (MRF)  model  based  multi-channel  Bayesian 
segmentation  technique.  The  HSI  color  space  was 
segmented  using  three  histogram  parameters.  An  iterative 
method  called  Iterated  Conditional  Modes  (ICM)  and  a 
simulated  annealing  are  employed  for  merging.  This 
proposed  scheme  is  computationally  intensive  compare  to 
[2]. 

In  this  paper,  the  segmentation  is  realized  in  two  stages. 
In  the  first  stage,  the  image  is  segmented  using  the 
homogeneity  and  intensity  values  of  the  pixels.  In  the 
second  stage,  sub-regions  are  obtained  by  differentiating  the 
pixels  on  the  basis  of  color  feature  hue  measure.  The  sub- 
regions  with  similar  color  are  then  merged. 

II.  Methodology 

A  2-D  histogram  in  the  intensity-homogeneity  domain  is 
obtained.  For  each  intensity  value,  the  number  of  pixels  with 
homogeneity  value  greater  than  a  certain  threshold  value  is 
computed.  The  image  is  segmented  into  regions  by  locating 
the  relative  minima  of  the  histogram.  These  regions  are 
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further  subdivided,  and  merged  using  color  feature  hue,  and 
CIE  (L*  a*  b*)  color  space  measure,  respectively. 

A.  Intensity  and  Homogeneity 

Let  (x,y)  is  the  location  of  a  pixel.  The  intensity,  I,  of  the 
pixel  (x,y)  is  computed  as  follows  [7]. 

I(x,  y)  =  1  /  3(R(x,  y)  +  G(x,  y)  +  B(x,  y))  (1) 

Suppose  I(x,  y)  is  the  intensity  of  a  pixel  P(x,  y)  at  the 

location  (x,y)  in  an  MxN  image,  w*','  is  a  size  dxd  window 

(2) 

centered  at  (x,y)  for  the  computation  of  variation,  w  is  a 

size  txt  window  centered  at  (x,y)  for  the  computation  for 
discontinuity,  d  and  t  are  odd  integers  greater  than  one.  The 

windows  and  wffy 1  are  considered  as  the  local  regions 

when  calculating  the  homogeneity  features. 

The  standard  deviation  of  a  pixel  P(x,  y)  is 


x+(d-l)/2  y+(d-l)/2 

yfcy)=  -T-  Yj  ^](I(p,q)-p(x,y))2  (2) 

\  “  p=x-(d-l)/2  q=y-(d-l)/2 

where  0  <  x,  p  <  M-l,  0  <  y,  q  <  N-l.  ji(x,  y)  is  the  mean  of 
the  gray  levels  within  window  given  by 

x+(d- 1)/2  y+(d- 1)/2 

nix,  y)  =  —y  'Ys  (3) 

“  p=x-(d- 1)/2  q=y-(d- 1)/2 

The  discontinuity  for  pixel  P(x,  y)  is  described  by  edge 
value.  There  are  many  different  edge  operators:  Sobel, 
Laplace,  Canny,  etc.  Here  as  it  is  not  required  to  find  the 
exact  locations  of  the  edges,  the  Sobel  operator  is  employed 
due  to  its  simplicity,  to  calculate  the  discontinuity.  The 
Sobel  operator  computes  the  magnitude  of  the  gradient  at 
location  (x,y)  using  [7]. 

e(x,y)  =  ^G2  +G2  (4) 

where  Gx  and  Gy  are  the  components  of  the  gradient  in  the  x 
and  y  directions,  respectively. 

The  standard  deviation  and  discontinuity  values  are 
normalized  in  order  to  achieve  computational  consistency. 


v(Kx,y))="(x-y) 

(5) 

V 

‘'max 

E(Kx,y))=e(x'y) 

(6) 

max 


where  Vmax  =  max{v(x,  y)}  ,  emax  =  max{e(x,  y)} ,  0  <  x  < 

M-l,  0  <  y  <  N-l. 

The  homogeneity  is  defined  as  [4] 

H(l(x,  y))=l- E(l(x,  y))x  V(l(x,  y))  (7) 

where  0  <  x  <  M-l,  0  <  y  <  N-l. 

The  value  of  the  homogeneity  at  each  location  of  an  image 
has  a  range  from  [0,1].  The  more  uniform  the  local  region 
surrounding  a  pixel  is,  the  larger  the  homogeneity  value  the 
pixel  has. 

B.  Segmentation  from  2-D  Histogram 

The  intensity  I(x,y)  and  homogeneity  H(x,y)  are  discretized 
with  i-th  value  of  intensity  and  j-th  value  of  homogeneity 
denoted  respectively  by  I;  and  H,.  Define  a  set  of  Rj,  given 
by 

Ry  ={P(x,y)|I(x,y)  =  Ii,H(x,y)  =  HJ)  (8) 

Let  n;j  be  the  number  of  pixels  in  Rjj.  A  2-D  histogram  is 
merely  a  plot  of  n;i  versus  Ij  and  Hj.  A  1-D  histogram  is 
obtained  from  the  2-D  histogram  as  follows.  Define  a  set  of 
R;  given  by 

Ri  =  {P(x,  y)  |  I(x,  y)  =  Ij ,  H(x,  y)  >  £>}  (9) 

where  8  is  the  threshold,  and  usually  chosen  close  to  the 
maximum  value  of  the  homogeneity,  say  8=0.95.  Let  n,  be 
the  number  of  pixels  in  Rj.  The  1-D  histogram  denoted  by 
h(i)  is  a  plot  of  n;  versus  I;.  The  relative  maxima  of  this 
histogram  are  computed  to  obtain  the  segments  using  a 
peak-finding  algorithm. 

The  peak-finding  algorithm: 

1.  Find  all  peaks'.  Find  the  set  of  points  corresponding  to 
the  local  maximums  of  the  histogram 

P0  =  {(i,  h(i))  |  h(i)  >  h(i  - 1)  &  h(i)  >  h(i  + 1),1  <  i  <  254} 

(10) 

2.  Find  significant  peaks'.  The  points  in  set  P0  form  a  new 
curve.  On  this  new  curve,  repeat  the  operation  of  step  1 . 
The  result  forms  set  P]. 

Pi  =  {(Pi,h(pi))  |  h(pi )  >  h(pM)  & h(pi)  >  h(pi+1),pi  e  P0} 

(11) 

All  the  points  in  set  P!  are  much  more  significant  than 
the  points  in  set  P0  in  determination  of  the  peaks  of  the 
histogram. 

3.  Thresholding',  includes  three  steps. 

The  first  step  is  to  remove  small  peaks.  If  a  peak  is  too 
small  compared  to  the  biggest  peak,  then  it  is  removed. 


Suppose  imax  is  the  value  of  the  highest  peak  satisfying 
b max  =  h(imax).  For  any  peak  j,  if  (h(j)/hmax)<0.05,  then 
peak  j  is  removed.  The  second  step  is  to  choose  one  peak 
if  two  peaks  are  too  close.  For  two  peaks  h(px )  and  h(p2), 
p2>pi,  if  p2-pi<15,  then  h=max{h(pi),h(p2)}.  Thus,  the 
peak  with  the  bigger  value  is  chosen.  The  third  step  is  to 
remove  a  peak  if  the  valley  between  two  peaks  is  not 
present.  The  presence  or  an  absence  of  a  valley  is 
determined  by  calculating  the  average  of  the  intensity 
values  between  the  two  peaks.  The  average  value  is 
computed  over  the  pixels  in  the  region  between  the 
peaks  pi  and  p2  as  follows  [4], 


Favg 


Pi  “P2 

5>(Pi) 

Pi=Pi 

p2  -Pi  +1 


(12) 


The  valley  between  the  two  peaks  is  considered  absent  if 
the  average  value,  havg  is  too  big  compared  to  the  peaks. 
That  is  if 


The  given  image  is  segmented  into  regions  based  on  similar 
intensities  and  hues. 

D.  Merging  the  Subdivisions 

The  segments  with  different  value  of  intensity  and  hue  but 
having  similar  colors  are  merged.  The  measure  of  color  is 
based  on  the  recommendation  of  CIE.  The  CIE  color  system 
is  obtained  from  the  nonlinear  transformation  of  three 
primary  colors  denoted  as  X,  Y,  and  Z,  which  are  obtained 
from  a  linear  combination  of  R,  G,  and  B  values.  The  CIE 
color  system  has  an  ability  to  measure  a  small  color 
difference  compare  to  RGB.  The  CIE  color  values  computed 
as  follows. 

Average  R,  G,  and  B  values  are  given  to  every  pixel 
belonging  to  the  same  segment,  Sy. 

R  =  E[R(x,  y)] 

G  =  E[G(x,  y)] 

B  =  E[B(x,  y)] 


havg  /((h(Pi)  +  h(p2))/ 2)  >  0.75  (13) 

The  smaller  of  the  two  peaks  for  which  the  valley  is  absent 
will  not  be  considered. 

This  peak-finding  algorithm  locates  the  globally 
significant  peaks  of  the  histogram.  After  the  peaks  are 
found,  the  valleys  are  determined  from  the  minimum 
between  any  two  adjacent  peaks.  The  valleys  thus  obtained 
serve  as  the  boundaries  for  the  segmentation  in  homogeneity 
domain. 

Let  v(k) ,  k=l,  2,  ...,  1  be  the  intensity  value  of  the 
valley.  Using  v(k)  as  the  threshold  value,  the  image  is 
segmented  as 

S i (k)  =  {(x,  y)  |  v(k - 1)  <  I(x,  y)  <  v(k)},  k  =  1 ,2,...,  1 

(14) 


C.  Subdivision  based  on  the  Color  Feature  Hue 

The  Hue,  H,  of  the  pixel  (x,y)  is  computed  as  follows  [6]. 

V3(G(x,y)  — B(x,y)) 


H(x,  y)  =  arctan 


(R(x,  y)  -  G(x,  y))+  (R(x,  y)  -  B(x,  y)) 


(15) 


Procedure  explained  in  the  previous  stage  is  adopted. 


where  E[.]  denotes  average  value  computed  over  all  pixels 
in  the  same  segment  Sy.  The  X,  Y,  and  Z  are  computed  as 
follows  [6]. 


"x' 

0.490  0.310  0.200 

R 

Y 

= 

0.177  0.813  0.011 

G 

(18) 

Z 

0.000  0.010  0.990 

B 

Instead  of  using  X,  Y,  and  Z  color  scheme,  a  color  scheme, 
which  possess  more  uniform  perceptual  properties,  denoted 
L*,  a*,  and  b*  is  employed.  L*,  a*,  and  b*  is  obtained 
through  a  nonlinear  transformation  on  X,  Y,  and  Z,  as 
follows  [6]. 


(19) 

(20) 

(21) 


where  (X0,  Y0,  Z0)  are  XYZ  values  for  the  standard  white. 

The  color  difference  between  the  any  two  pixels  in  the 
subdivisions  i,  and  j  are  computed  as  using  a  Euclidean 
distance,  given  by 


S1H  (i,  j )  =  {(x,  y )  e  Sj  (i)  |  Hth  (. j )  <  H(x,  y )  <  Hth  ( j  + 1)} 
i  =  0,1,2,...,  f-1;  j  =  0,1,2,...,  k-\ 


D  y  =  A/(LI-LJ)2+(ai-aJ)2+(b1-bJ)2  (22) 


(16) 


where  (L,  a,  by)  and  (Lj  a,  bj)  are  the  CIE  (L*  a  *  b*)  color 
representations  for  the  pixels  in  the  subdivisions  i,  and  j 


respectively.  There  will  be  total  of  (k(k-\))!2  such  color 
distance  measures.  Note  that  all  the  pixels  in  the  same 
subdivision  will  have  the  identical  color  value.  The  above 
Euclidean  distance  measure  matches  the  sensitivity  and 
perceptions  of  human  eyes. 

Based  on  this  color  scheme,  the  subdivisions  Sy  with  the 
similar  color  are  merged  using  the  Euclidean  distance 
measures  given  by  (22).  Let  k  be  the  total  number  of 
subdivisions,  which  is  also  the  total  number  of  CIE  color 
values. 

Let  |id  and  ad  be  the  average  and  the  standard  deviation 
respectively  of  the  color  differences  between  the  k 
subdivisions.  The  mean  and  the  standard  deviation  are  given 
by 


i  j 

(23) 

(24) 

where:  n  =  (k(k- 1)/2)  is  the  total  number  of,  |id  is  the  mean 
of  color  differences,  and  Dy  is  color  difference  between  i, 
and  j  given  by  the  equation  (22). 

The  regions  i,  and  j  will  be  merged  if 

.a 

H 

VI 

cT 

(25) 

where 

Th  =|id  -<7d 

(26) 

The  segments  {Sy}  having  similar  CIE  color  values  are 
merged  to  form  the  final  segment. 

Sk  =  USij’  k  =  1,2,...K 

i,je*r 

(27) 

where  the  index  set  K  is  defined  as 

H 

VI 

Q 

II 

Sc 

(28) 

III.  Results  and  discussion 

The  method  proposed  in  this  paper  has  been  tested  with  50 
clinical  endoscopic  images.  The  original  images  are  shown 
in  Fig.  1  and  the  segmentation  results  are  shown  in  Fig.  2. 
The  proposed  method  can  segment  the  image  into  regions, 
which  can  be  used  for  further  analysis  on  detecting  the 
abnormality  of  the  colon. 


Fig.  1.  Original  Clinical  Endoscopic  Images 


A 


Fig.  2.  Segmented  Endoscopic  Images 

IV.  Conclusion 

The  proposed  scheme  segments  the  image  based  on 
homogeneity,  color  feature  hue,  and  CIE  color  system.  The 
segmentation  algorithm  is  computationally  efficient  and 
may  be  implemented  in  real-time.  The  segmentation  results 
on  actual  images  were  promising  in  diagnosing 
abnormalities,  such  as  polyp,  ulcers,  and  bleeding. 
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